Adaptive Sensing in Uncertain Environments: Maximum Likelihood, Sensor Networks, and Reinforcement Learning

The advent of distributed and agile sensing systems that collect data in multiple locations and through a variety of sensing modalities has brought about new and exciting challenges to the field of signal processing. Motivated by problems that arise in the development of these systems, the thesis makes contributions in three domains: (1) distributed optimization for inference in sensor networks, (2) statistical tests for optimality that mitigate the problem of sensitivity to local maxima, and (3) development and analysis of reinforcement learning solutions to stochastic decision problems for resource allocation in agile sensing. A novel incremental gradient method, called incremental aggregated gradient (IAG), that can be used by wireless sensor networks to perform inference in a distributed manner, is proposed and analyzed. A gradient aggregation concept relaxes the common requirement of incremental methods for a diminishing step size for convergence, and a fast convergence rate is established. The convergence of IAG is established under a certain unimodality assumption. For non-convex problems however, for example when IAG is applied to find the maximum likelihood estimator, the method might stagnate at a local maximum. To mitigate this weakness, the following question is addressed: Given the location of a relative maximum of the log-likelihood function, how to assess whether it is the global maximum? We analyze and improve an existing statistical tool, called A Test for Global Maximum, that answers this question by posing it as a hypothesis